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ABSTRACT 

The National Center of Education Statistics conducted a mapping study that 
equated the percentage proficient or above on each state’s NCLB reading and 
mathematics tests in grades 4 and 8 to the NAEP scale. Each NAEP equivalent 
score was labeled according to NAEP’s achievement levels and used to com- 
pare state proficiency standards and to determine whether states have been 
raising their standards over time. An explicit purpose for the study was to use 
state proficiency scores to compare the effectiveness of schools across states, 
but the mapping study did not compare the rigor of a state’s standard with 
the overall achievement in the state. This paper examined statistically and 
graphically the relationship of the 2013 proficiency standard with NAEP 2013 
student achievement. The standard vs. achievement Pearson r correlation co- 
efficient in reading was 0.28 for Grade 4 and 0.01 for grade 8. The Pearson r 
in mathematics was 0.30 for both grades. There is no apparent reason to pos- 
tulate a relationship between student achievement and proficiency standards 
because student achievement is an outcome of pedagogical endeavor while 
proficiency standards are a product of political exercise. The rigor of a state’s 
proficiency standard has little relationship with overall student achievement 
in the state. (Contains 7 references, 1 table, and 5 figures.) 
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The National Center for Education Statistics has published a report entitled 
Mapping State Proficiency Standards onto NAEP Scales: Results from the 2013 
NAEP Reading and Mathematics Assessments (National Center for Education 
Statistics, 2015). For this paper, the report will be referred to as the mapping 
study. The executive summary stated the purpose for the study. 


Because [NCLB permitted] each state set its own standards, there 
was no assurance that students who met the standards of one 
state would be able to meet the standards of another state, and 
one could not compare the effectiveness of schools across states in 
terms of the percentages of students reported to meet the 
standards 

NCLB required NAEP participation for Title I. Knowing what per- 
centage of a state’s students performed at or above its cut point for 
proficiency on the state assessment, coupled with the state’s 
performance on NAEP, allowed the National Center for Education 
Statistics (NCES) to estimate where the expectation each state has 
for what students should learn or know falls on the NAEP scales — 
that is, NAEP provided a common scale on which the stringency of 
the various state criteria for proficiency could be compared. 

MAPPING STUDY RESULTS 

The mapping procedure generated a “NAEP equivalent score” for each state’s 
reading and mathematics proficiency standards at the fourth and eighth 
grades. The NAEP equivalent scores for each grade-subject were graphed by 
state from low to high. Figure 1, for example, illustrated the results for fourth 
grade reading. The graphs also indicated in which NAEP proficiency level the 
state’s NAEP equivalent score fell. 

States identified as Basic or as Proficient for each grade-subject were tallied. 
Finally, the tallies for 2013 were compared to counts from mapping studies 
conducted earlier using data from 2009 and 2011. The mapping study reported 
the following findings for fourth grade reading. 
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Figure 1. NAEP scale equivalents of state grade 4 reading standards for proficient 
performance, by state: 2013 



Stats 


SOURCE: U.S. Department erf Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of 
Educational Progress (NAEP), 2013 Reading Assessment 


Although the wide variation in standards persists, the number of 
states with grade 4 reading standards at or above the NAEP Basic 
level increased from 15 in 2009 and 20 in 2011 to 25 in 2013. 
Although in 2009 and 2011 no state standard was in the NAEP 
Proficient range, in 2013 two states had grade 4 reading standards 
in that range. In mathematics, the number of states with grade 4 
standards at or above the NAEP Basic level also increased, from 44 
in 2009 to 46 in 2011 and 47 in 2013, with five states having 
standards in the Proficient range in 2013 compared with one state 
each in 2009 and 2011. 

The mapping study used the same format to report results also for fourth grade 
mathematics and for eighth grade reading and mathematics. The results above 
are presented only to illustrate the objectives for the study: 

• To compare state proficiency standards by equating each state’s standard 
to a common scale, namely the NAEP scale. 

• To determine whether the states have been raising their proficiency 
standards over time. 
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DEFINITION OF NAEP ACHIEVEMENT LEVELS 

Since the mapping study used the NAEP achievement levels to label the rigor 
level of the state proficiency standards it is important to understand how the 
levels are defined. Table 1 provides some guidance (Stoneberg, 2013). It is 
important — even crucial — to understand that: 

• NAEP uses Proficient to name one achievement level. 

• NAEP uses proficiency in subject to define the Basic achievement level 

• NAEP Proficient does not mean proficiency in the subject. NAEP Basic 
estimates grade level expectations (similar to NCLB’s state proficient). 

Had the National Assessment Governing Board elected to use classroom grades 
from “A” to “F” instead of names for NAEP achievement levels, the Board might 
have used “A+” for Advanced, “A” for Proficient, and “B” and “C” for Basic. 

Table 1. English language descriptors and classroom grades that have been used to 
clarify the meaning of the NAEP achievement levels. 


HAEP 

Achievement 

Level 

English Language Descriptors Used to Explain 
the Meaning of the NAEP Achievement Levels 

Letter Grade (AT) 

Stoneberg 

(2007) 

Ravitch 

(2012) 

Advanced 


A to A+ 

A+ 

Proficient 

Some of the best students you know 
Many words and terms above grade level 
Mastery of complex material 
Higher than grade level performance 

8+ to A 

A 

Basic 

Proficiency in subject (common language meaning) 

C- to B 

B and C 

Overall understanding of grade appropriate text 
More than minimal competency 

Below Basic 

Minimal competency 

F to D+ 

Concern 
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EFFECTIVENESS OF SCHOOLS ACROSS STATES 

The expressed justification for the mapping study was to enable comparisons of 
the effectiveness of schools across states. The study, however, stopped 
short of comparing the states’ actual achievement scores with the rigor of their 
proficiency standards for reading and mathematics. Figures 2 through 5 
compare the states’ 2013 NAEP equivalent scores generated by the mapping 
study with the states’ overall student achievement reported by the 2013 NAEP 
assessments of reading and mathematics in grades 4 and 8. 

The standards vs. achievement Pearson r correlation coefficient in reading was 
0.28 for fourth grade and 0.01 for eighth grade. The Pearson rin mathematics 
was 0.30 for the fourth and eighth grades. 

There is little relationship between student achievement in a state and the 
state’s proficiency standard. Neither statistic appears to be useful for predicting 
or estimating the other. Actually, there is no logical reason to postulate a 
relationship. Student achievement is an outcome of pedagogical activity. State 
proficiency standards are a product of political exercise. 
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Figure 2. Two views comparing the NAEP equivalent score for a state’s 2013 fourth 
grade reading proficiency standard with the state’s NAEP average reading score on the 
2013 fourth grade assessment. Top view is ordered by rigor of reading proficiency 
standard; bottom view is ordered by achievement (average reading score). 
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Figure 3. Two views comparing the NAEP equivalent score for a state’s 2013 eighth 
grade reading proficiency standard with the state’s NAEP average reading score on the 
2013 eighth grade assessment. Top view is ordered by rigor of reading proficiency 
standard; bottom view is ordered by achievement (average reading score). 
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Figure 4. Two views comparing the NAEP equivalent score for a state’s 2013 fourth 
grade mathematics proficiency standard with the state’s NAEP average mathematics 
score on the 2013 fourth grade assessment. Top view is ordered by rigor of 
mathematics proficiency standard; bottom view is ordered by achievement (average 
mathematics score). 
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Figure 5. Two views comparing the NAEP equivalent score for a state’s 2013 eighth 
grade mathematics proficiency standard with the state’s NAEP average mathematics 
score on the 2013 eighth grade assessment. Top view is ordered by rigor of 
mathematics proficiency standard; bottom view is ordered by achievement (average 
mathematics score). 
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POLITICAL REPORTING OF MAP STUDY 


The chairman of the John Lock Foundation, for example, celebrated the 
mapping study from a political point of view. The John Locke Foundation was 
created in 1990 as an independent, 501(c)(3) research institute to work for 
truth, for freedom, and for the future of North Carolina. 

Texas, Massachusetts, Wisconsin, New York, and North Carolina 
are in an exclusive club. Can you guess its membership policy? 

[...] According to a recent report from the National Assessment of 
Educational Progress, these five states have the highest 
expectations for student proficiency in the country. [...] All other 
jurisdictions were found wanting. A majority didn’t even demand 
basic skills in 4th-grade reading. Six states — Alabama, 
Connecticut, Georgia, Idaho, Maryland, and Ohio — set standards 
so low that students in at least one of the two tested grades can 
rate as proficient on state reading and math exams despite the fact 
that they test “below basic” on NAEP exams (Hood, 2015). 

It is noteworthy that all 50 states scored at or above the cut-score for NAEP 
Basic (i.e., met NAEP’s grade-level expectation) on the 2013 NAEP assessments 
of reading and mathematics at grades 4 and 8, except for New Mexico (within 
measurement error of the cut-score) on the Grade 4 reading assessment. 

What happened when actual reading achievement results on the Grade 4 NAEP 
2013 assessment for the “highest expectations” state of North Carolina were 
compared with those of the six states that set their standards “so low”? On the 
2013 NAEP reading assessment, North Carolina fourth graders did score higher 
than their peers in Alabama. On the other hand, Connecticut and Maryland 
scored higher than North Carolina, and the reading scores for Georgia, Idaho 
and Ohio were not significantly different from North Carolina. 

CONCLUSION 

The rigor of a state’s proficiency standard had little to do with the overall 
student achievement in the state, not statistically and not logically. 
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